Monaural Multi-Talker Speech Recognition using Factorial Speech Processing Models

摘要

A Pascal challenge entitled monaural multi-talker speech recognition wasdeveloped, targeting the problem of robust automatic speech recognition againstspeech like noises which significantly degrades the performance of automaticspeech recognition systems. In this challenge, two competing speakers say asimple command simultaneously and the objective is to recognize speech of thetarget speaker. Surprisingly during the challenge, a team from IBM research,could achieve a performance better than human listeners on this task. Theproposed method of the IBM team, consist of an intermediate speech separationand then a single-talker speech recognition. This paper reconsiders the task ofthis challenge based on gain adapted factorial speech processing models. Itdevelops a joint-token passing algorithm for direct utterance decoding of bothtarget and masker speakers, simultaneously. Comparing it to the challengewinner, it uses maximum uncertainty during the decoding which cannot be used inthe past two-phased method. It provides detailed derivation of inference onthese models based on general inference procedures of probabilistic graphicalmodels. As another improvement, it uses deep neural networks for joint-speakeridentification and gain estimation which makes these two steps easier thanbefore producing competitive results for these steps. The proposed method ofthis work outperforms past super-human results and even the results wereachieved recently by Microsoft research, using deep neural networks. Itachieved 5.5% absolute task performance improvement compared to the firstsuper-human system and 2.7% absolute task performance improvement compared toits recent competitor.